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Methodology 


From July 15 to October 13, 2015, Hart Research conducted an online survey 
among 325 chief academic officers or designated representatives at AAC&U 
member institutions to explore how higher education institutions today are defining 
common learning outcomes and to document priorities and trends related to 
general education, equity, and emerging teaching practices. 


The margin of error is +4.4 percentage points for the entire sample, and it is larger 
for subgroups. The total population for the survey included 1,001 AAC&U member 
institutions that were invited to complete the survey, and thus the response rate for 
the survey is 32%. The sample is representative of AAC&U’s total membership in 
terms of both institution type (11% associates, 30% bachelor’s, 39% master’s, 
19% doctoral/research, and 1% other) and affiliation (46% public, 53% 
independent/religious, and 1% proprietary). 


Select findings from the survey about trends in learning outcomes assessment are 
outlined in this report. 


In addition, see other reports summarizing additional findings from the survey: 


e Bringing Equity and _ Quality Learning Together: Institutional 
Priorities for Tracking and Advancing Underserved Students’ Success 
(released November 16, 2015) 


e Recent Trends in General Education Design, Learning Outcomes, and 
Teaching Approaches (released January 19, 2016) 


See: http://www.aacu.org/about/2015-membersurvey 
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Overview of Key Findings 


Institutional Learning Outcomes: 


Nearly all AAC&U member institutions have a common set of learning 
outcomes for all of their undergraduate students. 

There is significant agreement across AAC&U member institutions about 
the learning outcomes they have adopted for all students. These 
outcomes apply to a broad range of skills and knowledge areas. 


Approaches to Assessing Student Learning Outcomes: 


The proportion of AAC&U member institutions assessing learning 
outcomes both in general education and more broadly at the 
institutional level has increased from six years ago. Majorities say they 
are assessing in each of these ways, with many assessing learning 
outcomes at both the department level and in general education. 

AAC&U member institutions that assess outcomes in general education 
use a variety of tools to do so. 

Among those who assess outcomes in general education, the most 
commonly used approach to assessing outcomes is the use of rubrics 
applied to samples of student work (91% of those assessing general 
education outcomes) and capstone projects (78%). 

Among AAC&U member institutions that assess outcomes in general 
education, 42% are using AAC&U’s VALUE (Valid Assessment of 
Learning in Undergraduate Education) rubrics. Furthermore, among 
those who use institutionally created rubrics, well over half (58%) say 
the VALUE rubrics informed the development of locally created rubrics. 


Use of E-Portfolios: 


Many institutions are using e-portfolios, but only a limited number of 
institutions require their use. There has been a slight uptick in the 
proportion of those that require the use of e-portfolios in the past seven 
years. 
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Key Findings 


1 Institutional Learning Outcomes: Most AAC&U member institutions 
have a common set of learning outcomes for all of their undergraduate 
students. 


Fully 85% of AAC&U member institutions report that they have a common set of 
intended learning outcomes for a// undergraduate students. This is up from 78% of 
institutions that said they had common learning outcomes in 2008. In addition, 
70% of institutions are tracking students’ achievement of these learning outcomes. 


Does your institution have a common set of intended 
learning goals or learning outcomes that apply to ALL 
undergraduate students? 


m Yes, have common set of intended learning goals/outcomes No, do not have 


85% 


78% 
22% 
15% 


November/December 2008 July — October 2015 


Baccalaureate (88%) and master’s institutions (87%) are slightly more likely than 
doctoral/research institutions (79%) to have a common set of learning outcomes 
that apply to all students. Religious institutions (92%) also are slightly more likely 
than public (82%) and non-religious independent institutions (84%) to have them. 


All AAC&U member institutions have specified intended learning outcomes in at 
least a few of their departments, including fully 67% who say that a// departments 
have them and another 27% who say most departments do. In 2008, a comparable 
65% of member institutions said they had defined outcomes in all departments. 
There is little difference by Carnegie Classification or Affiliation. 


Consensus on Broad Skills and Knowledge Areas: There is significant 
agreement across AAC&U member institutions about the learning 


outcomes they have adopted for all students. These outcomes apply to a 
broad range of skills and knowledge areas. 


AAC&U member institutions’ common learning outcomes address a broad range of 
skills and knowledge areas. Across the 11 skills and 11 knowledge areas measured 
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in the survey, majorities of member institutions who have a common set of learning 
outcomes say their learning outcomes address 18 of the 22 areas. 


due) Solad(ols mem Ga-idiseid(elal-M Mal-imat-\\-mm-t-leallacemeleinoclal- Mie] m\ 1 
Students That Address Specific Skills and Knowledge Areas 


(among institutions that have a common set of 
learning outcomes for all students)' 


2008 2015 


% % 
Intellectual Skills 
Writing skills 99 99 
Critical thinking and analytic reasoning skills 95 98 
Quantitative reasoning skills Oi 94 
Oral communication skills 88 82 
Intercultural skills and abilities 79 WS | 
Information literacy skills 76 76 
Ethical reasoning skills Pas. 75 
Specific Areas of Knowledge 
Knowledge of science 91 92 
Knowledge of mathematics 87 92 
Knowledge of humanities 92 92 
Knowledge of global world cultures 87 89 
Knowledge of social sciences 90 89 
Knowledge of the arts N/A 85 
Knowledge of diversity in the United States 73 73 
Knowledge of technology 61 49 
Knowledge of languages other than English 42 48 
Knowledge of American history 49 47 
Knowledge of sustainability 24 27 
Integration and Application of Knowledge and Skills 

Research skills and projects 65 75 
Integration of learning across disciplines 63 68 
Application of learning beyond the classroom 66 65 
Civic engagement and competence 68 63 


Among those who say their institution has a common set of intended learning 
outcomes, the area in which there has been the most notable decline since 2008 is 
in the proportion who say their learning outcomes address the area of technology 
(down from 61% in 2008 to 49% today). Conversely, the most notable increase has 
been among those who say their learning outcomes address research skills and 
projects (up from 65% in 2008 to 75% today). 


* Please see Appendix for results on learning outcomes that address each area of intellectual skill or ability among 
all AAC&U members. 
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Assessment of Learning Outcomes for Undergraduate Students: Today, 
more AAC&U member institutions are assessing learning outcomes 


across the curriculum and in general education than were six years ago. 
Majorities say they are assessing in each of these ways, but the share 
assessing learning outcomes across the curriculum (87%) continues to be 
larger than the share doing so in general education (67%). Today, 63% 
are assessing at both the department level and in general education. 


Fully 87% of AAC&U member institutions assess learning outcomes across the 
curriculum, which is up from the 72% who said they did so in 2009. Another 11% 
of administrators say their institution plans these types of assessments, while only 
2% say they are not doing it now and have no plans to do so. 


Fully 89% of AAC&U member institutions that have a common set of learning 
outcomes assess outcomes across the curriculum. The proportion is lower (75%) 
among institutions that do not have a common set of learning outcomes. 


Vy 


The large majority (85%) of member institutions say they assess students 
cumulative learning outcomes in departments, including 40% who say they do so in 
all departments. This is up from 27% who said they assessed in a// departments in 
2008. 


Proportions of Member Institutions that Assess 
Learning Outcomes in Departments 


Does your institution assess learning outcomes across the curriculum? If it does, 
are students’ cumulative learning outcomes assessed in departments? 


2008 2015 


87% 


A few/No 10% 


72% departments 


85% assess 
within 
-- departments 


A few/No 
departments 


1 0, 
ve 68% assess 
within 
departments 


28% 


13% 


Planning 


All/Most to do so All/Most 
departments 24% departments Planning to 
11% 
Assess Do not assess Assess Do not assess 
learning learning learning learning 
outcomes outcomes outcomes outcomes 


More than two in three (67%) member institutions indicate that they are assessing 
cumulative learning outcomes in general education across multiple courses, which 
is up from 52% who said they were doing this six years ago. Of the remaining 33% 
who do not currently assess cumulative learning outcomes in general education, 
most (25%) say they are planning to do so, while only 8% do not and have no 
plans to do so. 
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Assessing Cumulative Learning Outcomes in 
General Education 


Does your institution assess cumulative learning outcomes in general education 
across multiple courses? 


November — December 2008 July — October 2015 


67% 


52% 48% 


Planning 


ioe [ore] 
42% 


ia Valali ale) 


ioe [o<xe) 
25% 


Assess’ Do not assess Assess’ Donotassess 
learning learning learning learning 
outcomes outcomes outcomes outcomes 


Approaches for Assessing Student Learning Outcomes: AAC&U member 
institutions that assess outcomes in general education use a variety of 
tools to do so. 


Among member institutions that report assessing outcomes in general education, 
the approaches they use to assess outcomes most commonly include institutionally 
created rubrics applied to samples of student work (87%) and capstone projects 
(78%). More than two in three of those assessing learning outcomes in general 
education do so through student surveys and self-reports (64%) and _ locally 
developed common assignments in some courses (62%). Other tools used by 
these member institutions include locally developed examinations (46%), AAC&U 
VALUE rubrics applied to samples of student work (42%), standardized national 
tests of general skills (38%), and standardized national tests of general knowledge 
(33%). 
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Member institutions use a variety of methods 
to assess student learning outcomes in general 


education. 


Does your institution assess cumulative learning outcomes in general education 


across multiple courses? 


67% 


irate lalaliare] 


to do so 
25% 


Assess Donotassess 
learning learning 
outcomes outcomes 


Which of these does your institution use? 


Institutionally created rubrics applied 
to samples of student work 


Culminating or capstone projects 
Student surveys and self-reports 


Locally developed common assign- 
ments in some courses 


Locally developed examinations 


AAC&U VALUE rubrics applied to 
samples of student work 


Standardized national tests of general 
skills, such as critical thinking 


Standardized national tests of general 
knowledge, such as science or 
humanities 


87% 


78% 
64% 
62% 


46% 
42% 


38% 


33% 


Among institutions that assess outcomes in general education, 91% use rubrics 
applied to examples of student work (either institutionally created rubrics or AAC&U 
VALUE rubrics) for this assessment today, which is up from 77% who did so in 
2008. There also have been small increases in the share using capstone projects 
and locally developed common assignments in some courses. 


Institutions’ Use Of Methods to Assess Learning 
Outcomes in General Education 
(among institutions that assess cumulative 


learning outcomes in general education) 


Rubrics applied to examples of student work (Institutionally 


created or AAC&U VALUE rubrics) 77 
Culminating or capstone projects 70 
Student surveys and self-reports 67 
Locally developed common assignments in some courses 52 
Locally developed examinations 44 
Standardized national tests of general skills, such as critical 
thinking 49 
Standardized national tests of general knowledge, such as 

science or humanities 31 


33 


* Please see Appendix for results on use of approaches to assess learning outcomes among all AAC&U members. 
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AAC&U VALUE Rubrics: Among institutions assessing general 

education learning outcomes, 42% use AAC&U VALUE rubrics, which 
represents 28% of all AAC&U member institutions. Those that use VALUE 
rubrics are using a variety of them, particularly for critical thinking, written 
communication, quantitative literacy, and oral communication. Nearly 
one-third (32%) of those assessing institution-wide outcomes use VALUE 
rubrics to assess outcomes at the department level, which represents 27% 
of all AAC&U member institutions. 


AAC&U member institutions that use VALUE rubrics to assess student learning 
outcomes in general education are using them across a variety of areas, with 
rubrics for critical thinking (71%), written communication (69%), quantitative 
literacy (51%), and oral communication (50%) used most. 


VALUE Rubrics Used by Member Institutions to Assess 
Student Learning Outcomes in General Education 


(among institutions that use AAC&U VALUE rubrics to assess 
general education learning outcomes)? 


All 
Respondents 

% 
Critical thinking 71 
Written communication 69 
Quantitative literacy 5h 
Oral communication 50 
Information literacy 38 
Ethical reasoning 30 
Global learning 30 
Intercultural knowledge and competence 30 
Inquiry and analysis 29 
Civic engagement 29 
Problem solving 25 
Integrative learning 22 
Creative thinking 7 
Reading 13 
Foundations and skills for lifelong learning 
Teamwork 
I do not know which specific rubrics we are using 15 


Additionally, among institutions that use rubrics created at the institutional level, 
more than half (58%) say that AAC&U’s VALUE rubrics informed the development of 
their own rubrics a great deal or somewhat. 


° Please see Appendix for results on use of AAC&U VALUE rubrics among all AAC&U members. 
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E-Portfolios: Many AAC&U member institutions use’ electronic 
portfolios, but few institutions require them for all students. 


Today 64% of AAC&U member institutions say they use electronic portfolios for at 
least some students and programs, including 7% that require them for all students 
and 57% who use them for some students. This is up from 54% of member 
institutions who were using them for at least some students in 2008. (An additional 
23% of member institutions currently say they are exploring the feasibility of using 
e-portfolios. Only 13% say they have no plans to develop them.) 


Use of Electronic Portfolios 


In regard to electronic portfolios, which describes your campus? 


= Required of almost ™ Used for some students © Not used/exploring ™ Not used/no 
all students and programs feasibility of using plans to use 


November— December 2008 July — October 2015 
(57% required/used) (64% required/used) 
14% 3% 13% 7% 


23% 
54% 


29% 


57% 


Doctoral/research institutions (80%) are the most likely to require at least some 
students to use e-portfolios, compared with 70% of master’s institutions and 54% 
of baccalaureate institutions. For each type of institution, fewer than one in 10 says 
they require all students to use them. 


In the 2014 survey of employers, fully 80% said that when evaluating a job 
candidate, it would be helpful to have access to an electronic portfolio summarizing 
and demonstrating the individual’s accomplishments in key skill and knowledge 
areas, in addition to a résumé and college transcript. 


Final Note 


This report is the third in a series of three reports AAC&U is issuing in 2016 
summarizing findings from this survey of a representative sample of chief academic 
officers at AAC&U member institutions. These reports build on two earlier reports 
issued in 2009 and document the ways in which AAC&U member institutions are 
working on the educational reform goals articulated in AAC&U’s current strategic 
plan, Big Questions, Urgent Challenges: Liberal Education and America’s Global 
Future. 


For more information on previous surveys and the full text of AAC&U’s strategic 
plan, see www.aacu.org. 
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Appendix 1 
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Students That Address Specific Skills and Knowledge Areas 


(among all institutions) 


Intellectual Skills 


Writing skills 77 84 
Critical thinking and analytic reasoning skills 74 84 
Quantitative reasoning skills 71 80 
Oral communication skills 69 70 
Intercultural skills and abilities 62 68 
Information literacy skills 59 65 
Ethical reasoning skills 59 64 
Specific Areas of Knowledge 
Knowledge of science 71 78 
Knowledge of mathematics 68 78 
Knowledge of humanities 72 78 
Knowledge of global world cultures 68 76 
Knowledge of social sciences 70 76 
Knowledge of the arts N/A 72 
Knowledge of diversity in the United States 57 62 
Knowledge of technology 48 42 
Knowledge of languages other than English 33 41 
Knowledge of American history 39 40 
Knowledge of sustainability 18 23 
Integration and Application of Knowledge and Skills 
Research skills and projects 5 64 
Integration of learning across disciplines 49 57 
Application of learning beyond the classroom Sz 55 
Civic engagement and competence 53 54 
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Appendix 2 


Institutions’ Use of Methods to Assess Learning 
Outcomes In General Education 


(among all institutions) 


Rubrics applied to examples of student work 


(Institutionally created or AAC&U VALUE rubrics) 40 61 
Culminating or capstone projects 37 52 
Student surveys and self-reports 35 43 
Locally developed common assignments in some courses 27 42 
Locally developed examinations 23 31 
Standardized national tests of general skills, such as critical 

thinking 26 26 
Standardized national tests of general knowledge, such as 

science or humanities 16 22 
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Appendix 3 


VALUE Rubrics Used by Member Institutions to Assess 


Student Learning Outcomes in General Education 
(among all institutions) 


All 
Respondents 

% 
Critical thinking 20 
Written communication 19 
Quantitative literacy 14 
Oral communication 14 
Information literacy ol 
Ethical reasoning 8 
Global learning 8 
Intercultural knowledge and competence 8 
Inquiry and analysis 8 
Civic engagement 8 
Problem solving 7 
Integrative learning 6 
Creative thinking 5 
Reading 4 
Foundations and skills for lifelong learning 2 
Teamwork 2 
I do not know which specific rubrics we are using 4 
Do not use AAC&U VALUE rubrics 72 
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